United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
I nilid Stall-, Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



I0/730.HK4 



FILING DATE 



12/10/20(0 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



27572 7590 06/07/2010 

HARNESS, DICKEY & PIERCE, P.L.C. 
P.O. BOX 828 

BLOOMFIELD HILLS, MI 48303 



DURAN, ARTHUR D 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



l/ffflrC? nVrliUli Otfff Iff ids y 


Application No. 

10/736,884 


Applicant(s) 

FUKUI, YOSHIKI 


Examiner 

Arthur Duran 


Art Unit 

3622 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )KI Responsive to communication(s) filed on 21 May 2010 . 
2a )^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-11 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

Claim(s) is/are allowed. 

6) ^ Claim(s) 1-11 is/are rejected. 

7) 0 Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1) ^| Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26 d (Rev e 08-06r 



Office Action Summary 



Part of Paper No./Mail Date 20100602 



Application/Control Number: 10/736,884 
Art Unit: 3622 



Page 2 



DETAILED ACTION 

Claims 1-11 have been examined. 

Response to Amendment 

The Amendment filed 6/2/10 on is insufficient to overcome the prior rejection. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed 
or described as set forth in section 102 of this title, if the differences between the 
subject matter sought to be patented and the prior art are such that the subject 
matter as a whole would have been obvious at the time the invention was made 
to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was 
made. 

Claims 1-11 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yoshifumi (JP 2002-290964) in view of Kadowaki (20020171637 ) in view of Berman 
(5,523,791). 

Claims 1, 2, 8, 9, 10, 11: Yoshifumi (Applicant's IDS dated 12/16/2003) discloses 
many relevant features to the Applicant's claims (Applicant's Specification, Background 
Section [2, 7, 8, 10]). Yoshifumi further discloses an information presentation system 
comprising: 

a reflected-image displaying medium for displaying a reflected image of a 
peripheral surroundings that will be reflected in a mirror when a target person to whom 
information is presented looks at the mirror; 
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presentation-information storing means for storing presentation information to be 
presented; and 

information displaying means for displaying the information in the reflected-image 
displaying medium based on the presentation information stored in the presentation- 
information storing means; 

wherein the information displaying means superimposes the information on the 
reflected image displayed in the reflected-image displaying medium to display the 
superimposed information as part of the image, based on the presentation information 
stored in the presentation-information storing means (Applicant's Specification 
concerning Yoshifumi [2, 7, 8, 10]; and, Yoshifumi, ([40, 20, 48,49,50]), claims 6-9). 

Yoshifumi further discloses that the information can be advertising ([40]). 

Also, notice in these citations that a variety of Internet, advertising, and video 
content can be integrated with the mirror image. Also, note that the superimposed 
content is integrated with the overall mirror image. 

Yoshifumi does not explicitly disclose that the background of the user is a key 
part of the image or that the advertising/content is integrated with the background 
carefully such as to look like part of the mirror image. 

However, Kadowaki discloses presenting advertising in an attractive manner to 
the user ([662]) and that a mirror image of a user including the background of the user 
can be presented (Figure 46; [707, 708-710]). Also, note that Kadowaki states that it is 
old and well known that the display systems can be regular image screen rather than 
hologram screens ([5-24]). 
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And, Berman discloses advertising or a variety of content overlayed with a main 
image such that the images are well integrated (Figure 1; col 1, lines 11-25; col 1, lines 
40-55; col 2, lines 15-40; col 3, lines 13-25; col 3, lines 39-col 4, line 15). Also, notice 
that Berman's invention can work with broadcast signals (col 2, lines 15-20). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to add Kadowaki's advertising and mirror image with 
background and Berman's advertising and overlaying of imagined/created content to 
actual content to Yoshifumi's mirror image with added advertising/content. One would 
have been motivated to do this in order to better present advertising/content in a 
manner of interest to the user. 

Additionally, on 5/21/10, Applicant added new features. 
Yoshifumi discloses "an image capturing device for capturing images of a target 
person looking at a mirror and a peripheral landscape around the mirror; 

a positional relationship detecting module for detecting a positional relationship 
between the target person and the mirror based on the captured images;... 

module and the positional relationship between the target person and the mirror." 
(Fig. 1; [13, 14, 25]). Note that in Yoshifumi that the cameras in Figure 1 capture the 
positional relationship between the object or person and the mirror. Note that the cited 
specification states how the mirror knows how far away or at what angle the person is. 

Yoshifumi does not explicitly disclose that the background of the user is a key 
part of the image or that the advertising/content is integrated with the background 
carefully such as to look like part of the mirror image. 
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However, it is obvious that when the camera of Fig. 1 of Yoshifumi captures the 
image of the user it will also capture the background behind the user. Mirrors 
standardly reflect what is in front of them. So, if there is mirror in front of a person with 
wallpaper or background behind a user, the mirror will reflect the mirror and the 
background scenery. Also, it is standard that a camera will also capture what is front of 
it. So, Yoshifumi's camera in Figure 1 will capture the user and the background behind 
the user. Hence, it is obvious that Yoshifumi's digital mirror with camera will show the 
user and the background behind the user. 

As a further example of this, Kadowaki discloses presenting advertising in an 
attractive manner to the user ([662]) and that a mirror image of a user including the 
background of the user can be presented (Figure 46; [707, 704-710]). Also, note that 
Kadowaki states that it is old and well known that the display systems can be regular 
image screen rather than hologram screens ([5-24]). 

And, Berman discloses advertising or a variety of content overlayed with a main 
image such that the images are well integrated (Figure 1; col 1, lines 11-25; col 1, lines 
40-55; col 2, lines 15-40; col 3, lines 13-25; col 3, lines 39-col 4, line 15). Also, notice 
that Berman's invention can work with broadcast signals (col 2, lines 15-20). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to add Kadowaki's advertising and mirror image with 
background and Berman's advertising and overlaying of imagined/created content to 
actual content to Yoshifumi's mirror image with added advertising/content. One would 
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have been motivated to do this in order to better present advertising/content in a 

manner of interest to the user. 

Claim 3: The combination of the prior art renders obvious the advertisement 
presentation system according to claim 2, and Yoshifumi further discloses that the 
reflected-image displaying medium further comprises displaying means in which an 
image can be displayed, the advertisement presentation system further comprises a 
plurality of landscape-image capturing means for capturing landscape images around 
the reflected-image displaying medium and mirror-image generating means for 
generating a mirror image of the peripheral landscape around the reflected-image 
displaying medium based on the landscape images captured with the landscape-image 
capturing means, and wherein the advertisement displaying means superimposes the 
advertisement image on the mirror image that is generated in the mirror-image 
generating means to display the superimposed advertisement image in the displaying 
means as part of the peripheral landscape, based on the advertising information stored 
in the advertising-information storing means (see the rejection above; also, further see 
Kadowaki (Figure 46; [707, 708-710, 662]). 

Claim 4: The combination of the prior art renders obvious the advertisement 
presentation system according to claim 3, and Yoshifumi further discloses positional- 
relationship detecting means for detecting positional relationship between the reflected- 
image displaying medium and the target person, wherein, when the reflected-image 
displaying medium is a plane mirror that faces a predetermined direction, the mirror- 
image generating means generates the mirror image of the peripheral landscape that 
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will be reflected in the plane mirror viewed from the target person based on the result 
detected with the positional-relationship detecting means (see the rejection above; 
also, note that Yoshifumi varies the mirror image based on the position of the user in 
order to present a realistic mirror image). 

Claim 5: The combination of the prior art renders obvious the advertisement 
presentation system according to claim 4, and Yoshifumi further discloses target- 
person-image capturing means for capturing an image of the target person, wherein 
the positional-relationship detecting means detects the positional relationship between 
the reflected-image displaying medium and the target person based on the image 
captured with the target-person-image capturing means (see the rejection above; also, 
note that Yoshifumi varies the mirror image based on the position of the user in order 
to present a realistic mirror image). 

Claim 6: The combination of the prior art renders obvious the advertisement 
presentation system according to claim 4, and Yoshifumi further discloses positional- 
information acquiring means for acquiring positional information of a communication 
terminal that the target person carries via communication with the communication 
terminal, wherein the positional-relationship detecting means detects the positional 
relationship between the reflected-image displaying medium and the target person 
based on the positional information acquired with the positional-information acquiring 
means (see the rejection above; also, note that Yoshifumi varies the mirror image 
based on the position of the user in order to present a realistic mirror image). 
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Claim 7: The combination of the prior art renders obvious the advertisement 
presentation system according to claim 3, and Yoshifumi further discloses that face- 
image capturing means for capturing a face image of a person around the reflected- 
image displaying medium and sight-line detecting means for detecting that the person 
turns his/her line of sight onto the reflected-image displaying medium based on the 
face image captured with face image capturing means, 

wherein the person who is detected turning his/her line of sight onto the 
reflected-image displaying medium with the sight-line detecting means is the target 
person and wherein, when the reflected-image displaying medium is the plane mirror 
that faces a predetermined direction, the mirror-image generating means generates the 
mirror image of the peripheral landscape that 

will be reflected in the plane mirror viewed from the target person (see the rejection 
above; also, note that Yoshifumi varies the mirror image based on the position and the 
facial angle of the user in order to present a realistic mirror image). 

Claims 1-11 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yoshifumi (JP 2002-290964) in view of Kadowaki (20020171637 ) in view of Machtig 
(6042235). 

Claims 1, 2, 8, 9, 10, 11: Yoshifumi (Applicant's IDS dated 12/16/2003) discloses 
many relevant features to the Applicant's claims (Applicant's Specification, Background 
Section [2, 7, 8, 10]). Yoshifumi further discloses an information presentation system 
comprising: 
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a reflected-image displaying medium for displaying a reflected image of a 
peripheral surroundings that will be reflected in a mirror when a target person to whom 
information is presented looks at the mirror; 

presentation-information storing means for storing presentation information to be 
presented; and 

information displaying means for displaying the information in the reflected-image 
displaying medium based on the presentation information stored in the presentation- 
information storing means; 

wherein the information displaying means superimposes the information on the 
reflected image displayed in the reflected-image displaying medium to display the 
superimposed information as part of the image, based on the presentation information 
stored in the presentation-information storing means (Applicant's Specification 
concerning Yoshifumi [2, 7, 8, 10]; and, Yoshifumi, ([40, 20, 48,49,50]), claims 6-9). 

Yoshifumi further discloses that the information can be advertising ([40]). 

Also, notice in these citations that a variety of Internet, advertising, and video 
content can be integrated with the mirror image. Also, note that the superimposed 
content is integrated with the overall mirror image. 

Yoshifumi does not explicitly disclose that the background of the user is a key 
part of the image or that the advertising/content is integrated with the background 
carefully such as to look like part of the mirror image. 

Additionally, on 5/21/10, Applicant added new features. 
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Note that the Machtig reference replaces the Official Notice related to Disney's 
Haunted Mansion in the action dated 2/26/10. On 5/21/10, Applicant challenged the 
Disney reference. Machtig's "pepper's ghost" technique describes the technique used 
by Disney. 

Yoshifumi discloses "an image capturing device for capturing images of a target 
person looking at a mirror and a peripheral landscape around the mirror; 

a positional relationship detecting module for detecting a positional relationship 
between the target person and the mirror based on the captured images;... 

module and the positional relationship between the target person and the mirror." 
(Fig. 1; [13, 14, 25]). Note that in Yoshifumi that the cameras in Figure 1 capture the 
positional relationship between the object or person and the mirror. Note that the cited 
specification states how the mirror knows how far away or at what angle the person is. 

Yoshifumi does not explicitly disclose that the background of the user is a key 
part of the image or that the advertising/content is integrated with the background 
carefully such as to look like part of the mirror image. 

However, it is obvious that when the camera of Fig. 1 of Yoshifumi captures the 
image of the user it will also capture the background behind the user. Mirrors 
standardly reflect what is in front of them. So, if there is mirror in front of a person with 
wallpaper or background behind a user, the mirror will reflect the mirror and the 
background scenery. Also, it is standard that a camera will also capture what is front of 
it. So, Yoshifumi's camera in Figure 1 will capture the user and the background behind 
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the user. Hence, it is obvious that Yoshifumi's digital mirror with camera will show the 
user and the background behind the user. 

As a further example of this, Kadowaki discloses presenting advertising in an 
attractive manner to the user ([662]) and that a mirror image of a user including the 
background of the user can be presented (Figure 46; [707, 704-710]). Also, note that 
Kadowaki states that it is old and well known that the display systems can be regular 
image screen rather than hologram screens ([5-24]). 

And, Machtig discloses advertising (4:10-17) and that it is old and well known 
technique that a nonreal image can be superimposed or overlayed or inserted carefully 
into the scene of real user with a real background (3:17-45). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to add Kadowaki's advertising and mirror image with 
background and Machtig's advertising and overlaying of imagined/created content to 
actual content to Yoshifumi's mirror image with added advertising/content. One would 
have been motivated to do this in order to better present advertising/content in a 
manner of interest to the user. 

Claims 3-7: Please see the rationale for the rejection of the dependent claims 
above. 

Response to Arguments 

Applicant's arguments with respect to the claims have been considered but are 
not found persuasive. 

On 5/21/10, Applicant presented arguments related to the following features. 
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On 5/21/10, Applicant added the following new features to the independent claim 

1: 

"an image capturing device for capturing images of a target person looking at a 
mirror and a peripheral landscape around the mirror; 

a positional relationship detecting module for detecting a positional relationship 
between the target person and the mirror based on the captured images;... 

module and the positional relationship between the target person and the mirror." 
Yoshifumi discloses "an image capturing device for capturing images of a target 
person looking at a mirror and a peripheral landscape around the mirror; 

a positional relationship detecting module for detecting a positional relationship 
between the target person and the mirror based on the captured images;... 

module and the positional relationship between the target person and the mirror." 
(Fig. 1; [13, 14, 25]). Note that in Yoshifumi that the cameras in Figure 1 capture the 
positional relationship between the object or person and the mirror. Note that the cited 
specification states how the mirror knows how far away or at what angle the person is. 

Yoshifumi does not explicitly disclose that the background of the user is a key 
part of the image or that the advertising/content is integrated with the background 
carefully such as to look like part of the mirror image. 

However, it is obvious that when the camera of Fig. 1 of Yoshifumi captures the 
image of the user it will also capture the background behind the user. Mirrors 
standardly reflect what is in front of them. So, if there is mirror in front of a person with 
wallpaper or background behind a user, the mirror will reflect the mirror and the 
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background scenery. Also, it is standard that a camera will also capture what is front of 
it. So, Yoshifumi's camera in Figure 1 will capture the user and the background behind 
the user. Hence, it is obvious that Yoshifumi's digital mirror with camera will show the 
user and the background behind the user. 

As a further example of this, Kadowaki discloses presenting advertising in an 
attractive manner to the user ([662]) and that a mirror image of a user including the 
background of the user can be presented (Figure 46; [707, 704-710]). Also, note that 
Kadowaki states that it is old and well known that the display systems can be regular 
image screen rather than hologram screens ([5-24]). 

And, Berman discloses overlaying (see above). Alternatively, Machtig discloses 
advertising (4:10-17) and that it is old and well known technique that a nonreal image 
can be superimposed or overlayed or inserted carefully into the scene of real user with 
a real background (3:17-45). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to add Kadowaki's advertising and mirror image with 
background and Berman's or Machtig's advertising and overlaying of imagined/created 
content to actual content to Yoshifumi's mirror image with added advertising/content. 
One would have been motivated to do this in order to better present advertising/content 
in a manner of interest to the user. 

Note that the Machtig reference replaces the Official Notice related to Disney's 
Haunted Mansion in the action dated 2/26/10. On 5/21/10, Applicant challenged the 
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Disney reference. Machtig's "pepper's ghost" technique describes the technique used 
by Disney. 

Conclusion 

The following prior art made of record and not relied upon is considered pertinent 
to applicant's disclosure: 

a) Rosser (5,264,933) and Cristofalo (200201521 17) disclose overlaying of ads 
into images; Weinreich discloses mirror images with advertising added; Adriansen 
discloses integrating ad content and background content; Rottcher discloses 
intergrating a mirror image and ad content. 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Arthur Duran whose telephone number is (571)272- 
6718. The examiner can normally be reached on Mon- Fri, 8:00-4:30. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Eric Stamber can be reached on (571) 272-6724. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Arthur Duran 
Primary Examiner 
Art Unit 3622 

/Arthur Duran/ 

Primary Examiner, Art Unit 3622 
6/12/10 



